Fusing Audio, Textual and Visual Features for Sentiment Analysis of News Videos

This paper presents a novel approach to perform sentiment analysis of news videos, based on the fusion of audio, textual and visual clues extracted from their contents. The proposed approach aims at contributing to the semiodiscoursive study regarding the construction of the ethos (identity) of this media universe, which has become a central part of the modern-day lives of millions of people. To achieve this goal, we apply state-of-the-art computational methods for (1) automatic emotion recognition from facial expressions, (2) extraction of modulations in the participants' speeches and (3) sentiment analysis from the closed caption associated to the videos of interest. More specifically, we compute features, such as, visual intensities of recognized emotions, field sizes of participants, voicing probability, sound loudness, speech fundamental frequencies and the sentiment scores (polarities) from text sentences in the closed caption. Experimental results with a dataset containing 520 annotated news videos from three Brazilian and one American popular TV newscasts show that our approach achieves an accuracy of up to 84% in the sentiments (tension levels) classification task, thus demonstrating its high potential to be used by media analysts in several applications, especially, in the journalistic domain.Comment: 5 pages, 1 figure, International AAAI Conference on Web and Social Medi

A vision-based system to support tactical and physical analyses in futsal

This paper presents a vision-based system to support tactical and physical analyses of futsal teams. Most part of the current analyses in this sport are manually performed, while the existing solutions based on automatic approaches are frequently composed of costly and complex tools, developed for other kind of team sports, making it difficult their adoption by futsal teams. Our system, on the other hand, represents a simple yet efficient dedicated solution, which is based on the analyses of image sequences captured by a single stationary camera used to obtain top-view images of the entire court. We use adaptive background subtraction and blob analysis to detect players, as well as particle filters to track them in every video frame. The system determines the distance traveled by each player, his/her mean and maximum speeds, as well as generates heat maps that describe players’ occupancy during the match. To present the collected data, our system uses a specially developed mobile application. Experimental results with image sequences of an official match and a training match show that our system provides data with global mean tracking errors below 40 cm, demanding on 25 ms to process each frame and, thus, demonstrating its high application potential

Object-based Image Retrieval using Local Feature Extraction and Relevance Feedback

This paper addresses the problem of object-based image retrieval, by using local feature extraction and a relevance feedback mechanism for quickly narrowing down the image search process to the user needs. This approach relies on the hypothesis that semantically similar images are clustered in some feature space and, in this scenario: (i) computes image signatures that are invariant to scale and rotation using SIFT, (ii) calculates the vector of locally aggregated descriptors (VLAD) to make a fixed length descriptor for the images, (iii) reduce the VLAD descriptor dimensionality with Principal Component Analysis (PCA) and (iv) uses the k-Means algorithm for grouping images that are semantically similar. The proposed approach has been successfully validated using 33,192 images from the ALOI database, obtaining a mean recall value of 47.4 % for searches of images containing objects that are identical to the object query and 20.7 % for searches of images containing different objects (albeit visually similar) to the object query. General Terms Content-based image retrieval, relevance feedback, feature extraction. Keywords Object-based image retrieval, scale invariant feature transform, principal component analysis, vector of locally aggregated descriptors, clustering algorithms. 1

SAPTE : a multimedia information system to support the discourse analysis and information retrieval of television programs.

This paper presents a novel multimedia information system, called SAPTE, for supporting the discourse analysis and information retrieval of television programs from their corresponding video recordings. Unlike most common systems, SAPTE uses both content independent and dependent metadata, which are determined by the application of discourse analysis techniques as well as image and audio analysis methods. The proposed system was developed in partnership with the free-to-air Brazilian TV channel Rede Minas in an attempt to provide TV researchers with computational tools to assist their studies about this media universe. The system is based on the Matterhorn framework for managing video libraries, combining: (1) discourse analysis techniques for describing and indexing the videos, by considering aspects, such as, definitions of the subject of analysis, the nature of the speaker and the corpus of data resulting from the discourse; (2) a state of the art decoder software for large vocabulary continuous speech recognition, called Julius; (3) image and frequency domain techniques to compute visual signatures for the video recordings, containing color, shape and texture information; and (4) hashing and k-d tree methods for data indexing. The capabilities of SAPTE were successfully validated, as demonstrated by our experimental results, indicating that SAPTE is a promising computational tool for TV researchers